Generating Activity Summaries

ABSTRACT

A system and method for generating activity summaries for users. User activity information is received by a user activity information receiver module. Once enough data is received and processed, the data is analyzes and segmented to determine and create an activity summary or story. Content is selected. The selection of the content includes selection of content items, such as multimedia items, e.g. pictures and videos. Secondary information, such as user activity information or location information is analyzed. A story is generated based on the selected content, including the selected content. The story is represented by a display of the selected media and other information associated with the media.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority, under 35 U.S.C. §119, to U.S.Provisional Patent Application No. 62/001,027, filed May 20, 2014 andentitled “Generating Activity Summaries”; U.S. Provisional PatentApplication No. 62/001,029, filed May 20, 2014 and entitled “Layout forDisplaying Activity Summaries”; U.S. Provisional Patent Application No.62/001,038, filed May 20, 2014 and entitled “User Interface forDisplaying Activity Summaries”; and U.S. Provisional Patent ApplicationNo. 62/001,028, filed May 20, 2014 and entitled “Backend Pipeline forStory Generation”; all of which are incorporated by reference in theirentireties.

TECHNICAL FIELD

The present disclosure relates to generating activity summaries. Morespecifically, the present disclosure relates to generating activitysummaries based on user activity and multimedia content.

BACKGROUND

Social networks are becoming an increasingly popular way for people tostay connected. This increasing popularity of social networks has givenrise to many social network services that have developed various waysusers of the social network can communicate and share information. Userswithin a social network can send each other messages, monitor otherusers' activities on a daily basis and share personal information,including personal photographs and videos. Social networking serviceshave provided a great forum for users to remain in close contact despitegeographic distance or uncoordinated schedules. Further, the developmentof other online services that enable the general sharing of informationhas also increased. The ability for users to share information withothers, and view information about others is available in many differentvenues.

Users of social networks also participate in many activities or onlineactivities, such as watching videos, posting pictures, postingnotification of current user information or activity, or sharinglocation whereabouts. Within such venues of these social networks, usersare able determine what information they chose to share and how and towhom that information is shared. Typically, the information may beshared on a social network portal of a user as the user decides to sharethe information, but the shared information is typically not organizedin a meaningful or interesting way.

SUMMARY

The present disclosure relates to generating activity summaries. Moreparticularly, the present disclosure relates to generating activitysummaries using digitally recorded information and other information.Still more particularly, the present disclosure relates to generatingactivity summaries using multimedia content and user activityinformation.

In general, one innovative aspect of the subject matter described inthis disclosure may be embodied in methods that may include receivinguser activity information. The user activity information may includeactivity information of a user's online activity. Once enough data oractivity information is received, the data is analyzed and segmented todetermine and create an activity summary or a story. Content isselected. The selection of the content includes selection of contentitems, such as multimedia items, e.g. pictures and videos. Secondaryinformation, such as user activity information or location informationis analyzed. A story is generated based on the received user activityinformation.

In one implementation, the method includes enriching the content withadditional information about the content. In another implementation, themethod further includes assigning a title to the story.

In another implementation, the method may further include categorizingthe user activity information, generating an activity summary, whereinthe activity summary includes the categorized user activity information,and sending the activity summary for display.

These and other implementations may provide one or more of the followingfeatures. The method may further include grouping the categorized useractivity information in accordance with commonalities. The method mayalso include determining the beginning of a grouping. The method mayalso include ranking the grouped user activity information according torelevance to the user and/or user's contacts. Further the method mayinclude determining when the group is complete. The method may alsoinclude associating a verb identifier with a user activity andcategorizing the user activity information according to the verbidentifier. In some implementations, the activity summary includes morethan one type of user activity. In some implementations, the method mayinclude allowing the user to share the generating activity summary withother users.

These and other implementations may provide one or more of the followingfeatures. According to one innovative aspect of the subject matterdescribed in this disclosure, a computer program product comprises anon-transitory computer readable medium encoding instructions that, inresponse to execution by a computing device, cause the computing deviceto perform operations comprising receiving user activity information,categorizing the user activity information, generating an activitysummary, wherein the activity summary includes the categorized useractivity information and sending the activity summary for display. Theuser activity information includes activity information of a user'sonline activity.

These and other implementations may provide one or more of the followingfeatures. In some implementations, the instructions of the computerprogram product may further cause the computing device to furtherperform grouping categorized user activity information in accordancewith commonalities. In some implementations, the instructions of thecomputer program product may further cause the computing device tofurther perform ranking the grouped user activity information accordingto relevance to the user and/or user's contacts. The instructions of thecomputer program product may further cause the computing device tofurther perform determining when the group is complete. The instructionsof the computer program product may further cause the computing deviceto further perform associating a verb identifier with a user activityand categorizing the user activity information according to the verbidentifier. In some implementations, the instructions of the computerprogram product may further cause the computing device to furtherperform allowing the user to share the generating activity summary withother users. In some implementations the activity summary includes morethan one type of user activity.

A system for generating activity summaries for users is also disclosed.User activity information is received by a user activity informationreceiver module. The user activity information includes activityinformation of a user's online activity. The user activity informationis then categorized by a categorization module, which in someimplementations, also groups the categorized user activity informationin accordance with commonalities identified among the user activityinformation. In some implementations, the categorized user activityinformation is ranked according to relevance to the user by the rankingmodule or according to relevance to the user's contacts by thecategorization module. An output generation module 316 determines whenthe groupings are complete. Activity summaries are then generated by theoutput generation module 316. The activity summary includes thecategorized user activity information. The activity summary is sent fordisplay on a user device of a user.

These and other implementations may provide one or more of the followingfeatures. In some implementations, the categorization module furtherassociates a verb identifier with a user activity, and categorizes theuser activity information according to the verb identifier. In someimplementations, the activity summary generation module allows the userto share the generating activity summary with other users.

These and other implementations may provide one or more of the followingadvantages. Activity summaries may be generated that summarize useractivity information, or digitally recorded information, or inferredactivity. For example, activity information may be anything that isdigitally record or inferred and authorized by the user to analyze andshare. Such information may be based on a user's life and based on theirdigital footprints, for example, location tracks, search history, webhistory, check-ins, posts created, documents created, web pagesbookmarked, keep reminders made, photos and videos taken (and all theirmetadata like time of day and geocode), etc. Additionally, there may beinformation from content, such as multimedia (photos, videos) thatcontain information to provide an inference of time of day or landmarksor specific places (when geocode is missing). In some implementations,activity summaries may be generated that summarize the user's real-worldactivity. In some implementations, a mobile application is implementedon a mobile device that is carried by the user, and that application hasaccess to the phone's location, then activity can be directly inferred.These summaries may organize the user's activity, including onlineactivity and group them into interesting stories that convey the user'sexperiences, which may be helpful in conveying interesting lifeexperiences.

BRIEF DESCRIPTION OF THE DRAWINGS

The implementations are illustrated by way of example, and not by way oflimitation in the figures of the accompanying drawings in which likereference numerals are used to refer to similar elements.

FIG. 1 is a block diagram illustrating an example of a system forgenerating activity summaries.

FIG. 2 is a block diagram illustrating an example of a network serverdevice.

FIG. 3 is a block diagram illustrating an example of an activity summarygeneration module.

FIG. 4 is a flow chart illustrating an example method for generatingactivity summaries.

FIG. 5 is a flow chart illustrating an example method for generatingactivity summaries.

FIG. 6 is a flow chart illustrating another example method forgenerating activity summaries.

FIG. 7 is a flow chart illustrating an example method for performingdata analysis to create an activity summary.

FIG. 8 is an example graphical user interface showing a title page of anactivity summary.

FIG. 9 is an example graphical user interface showing another page of anactivity summary.

FIG. 10 is an example graphical user interface showing yet another pageof an activity summary.

FIG. 11 is an example graphical user interface showing another page ofan activity summary.

FIG. 12 is an example graphical user interface showing another exampleof a title page of an activity summary.

FIG. 13 is an example graphical user interface showing another page ofan activity summary.

FIG. 14 is an example graphical user interface showing yet another pageof an activity summary.

FIG. 15 is an example graphical user interface showing another page ofan activity summary.

DETAILED DESCRIPTION

A system and method for generating activity summaries is described. Anactivity summary may be a visual representation of a story. For example,an activity summary may be a collection of content, e.g., pictures,videos, text, specifically selected and organized to tell a story. Sucha collection of content may be supplemented with additional informationto further enhance and enrich the story.

In one implementation, an activity summary may be generated based onuser activity information and content generated from a single useractivity. In another implementation, an activity summary may begenerated based on user activity information and content generated froma grouping of multiple user activities. Examples of such user activityinclude interactions with content, e.g., watching videos, postingpictures, posting notification of current user information or activity,sharing location whereabouts, or other activity performed online. Insome cases, two or more activities can be categorized and groupedaccording to one or more commonalities. For example, videos a userwatched can be grouped together. As another example, activitiesassociated with a particular even a user has attended or trip the userhas taken (e.g., pictures taken on the trip, locations identified, orrestaurants visited during the trip) may be grouped together.

An activity summary may also be generated based on user activityinformation and/or content generated from multiple users. In thismanner, content from a second user may be used to tell the story of afirst user. For example, shared photos of an event from multiple usersmay be used to generate an activity summary.

In some non-limiting implementations, the system may suggest that theuser share the generated activity summary with one or more persons orgroups. This beneficially allows a user to display certain activityinformation in a way that is more interesting for the user's contacts toview.

In the following description, for purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the implementations. It will be apparent, however, thatthe implementations can be practiced without these specific details. Inother instances, structures and devices are shown in block diagram formin order to avoid obscuring the implementations. For example, thepresent disclosure is described in one implementation below withreference to user interfaces and particular hardware. However, thepresent disclosure applies to any type of computing device that canreceive data and commands, and any peripheral devices providingservices.

Reference in the specification to “one implementation” or “animplementation” means that a particular feature, structure, orcharacteristic described in connection with the implementation isincluded in at least one implementation. The appearances of the phrase“in one implementation” in various places in the specification are notnecessarily all referring to the same implementation.

Some portions of the detailed descriptions that follow are presented interms of algorithms and symbolic representations of operations on databits within a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

Some implementations also relate to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but is not limited to, any type ofdisk including floppy disks, optical disks, CD-ROMs, and magnetic disks,read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, flash memories including USB keyswith non-volatile memory or any type of media suitable for storingelectronic instructions, each coupled to a computer system bus.

Furthermore, implementations can take the form of a computer programproduct accessible from a non-transitory computer-usable orcomputer-readable medium providing program code for use by or inconnection with a computer or any instruction execution system. For thepurposes of this description, a computer-usable or computer readablemedium can be any apparatus that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

The medium can be an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system (or apparatus or device) or apropagation medium. Examples of a computer-readable medium include asemiconductor or solid state memory, magnetic tape, a removable computerdiskette, a random access memory (RAM), a read-only memory (ROM), arigid magnetic disk and an optical disk. Current examples of opticaldisks include compact disk-read only memory (CD-ROM), compactdisk-read/write (CD-R/W) and DVD.

A data processing system suitable for storing and/or executing programcode will include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution.

Input/output or I/O devices (including but not limited to keyboards,displays, pointing devices, etc.) can be coupled to the system eitherdirectly or through intervening I/O controllers.

Network adapters may also be coupled to the system to enable the dataprocessing system to become coupled to other data processing systems orremote printers or storage devices through intervening private or publicnetworks. Modems, cable modem and Ethernet cards are just a few of thecurrently available types of network adapters.

Finally, the algorithms and displays presented herein are not inherentlyrelated to any particular computer or other apparatus. Variousgeneral-purpose systems may be used with programs in accordance with theteachings herein, or it may prove convenient to construct morespecialized apparatus to perform the required method steps. The requiredstructure for a variety of these systems will appear from thedescription below. In addition, the present implementations are notdescribed with reference to any particular programming language. It willbe appreciated that a variety of programming languages may be used toimplement the teachings of the implementations as described herein.

FIG. 1 is a block diagram illustrating an example of a system 100 forgenerating activity summaries. The illustrated system 100 includes userdevices 115 a, 115 b, and 115 n (also referred to collectively as userdevices 115 or individually as user device 115) that are accessed byusers 125 a, 125 b, and 125 n (also referred to collectively as users125 or individually as user 125), a network server 101, an applicationserver 107, an e-mail server 123 and a search server 180. In theillustrated implementation, these entities are communicatively coupledvia a network 105. Although only three user devices 115 a/115 b/115 nare illustrated, any number of user devices 115 a/115 b/115 n areavailable to any number of users 125 a/125 b/125 n.

The illustrated implementations of a system 100 includes user devices115 a, 115 b that are accessed by users 125 a, 125 b, a network server101 that is coupled to data storage 110 and an application server 107.In the illustrated implementation, these entities are communicativelycoupled via a network 105. The user devices 115 a, 115 b, 115 n in FIG.1 are used by way of example. While FIG. 1 illustrates three devices,the present disclosure applies to any system architecture having one ormore user devices. Furthermore, while only one network 105 is coupled tothe user devices, 115 a, 115 b, 115 n the network server 101 and theapplication server 107, in practice any number of networks 105 can beconnected to the entities.

Although only one network server 101 is shown, it will be recognizedthat multiple servers may be present. In some embodiments, the networkserver may be a social network server. A social network is any type ofsocial structure where the users are connected by a common feature.Examples include, but are not limited to, blogs, microblogs and Internetforums. The common feature includes friendship, family, a commoninterest, etc.

The network server 101 may be coupled to the network 105 via signal line104. In some implementations, the network server 101 may include asocial network module 209 (as will be shown in FIG. 2). Although onlyone network server 101 is shown, persons of ordinary skill in the artwill recognize that multiple servers may be present. A social network isany type of social structure where the users are connected by a commonfeature. The common feature may include, without limitation, commoninterests, work connections, family and other relationship connections,e.g., friendship, family, etc. The common features are provided by oneor more social networking systems, such as those included in the system100, including explicitly-defined relationships and relationshipsimplied by social connections with other online users, where therelationships form a social graph. In some examples, a social graph canreflect a mapping of these users and how they are related. Furthermore,it should be understood that, in some implementations, the networkserver 101 and social network module 209 (FIG. 2) are representative ofone social network and that there may be multiple social networks (notshown) coupled to the network 105, each having its own server,application and social graph (not shown). For example, a first socialnetwork may be more directed to business networking, a second moredirected to or centered on academics, a third is more directed to localbusiness, a fourth directed to dating and others of general interest ora specific focus.

As shown in FIG. 1, network server 101, may include, among various othermodules, an activity summary generation module 220 a.

Additionally or alternatively, an activity summary module 220 b may beincluded in the application server 107. For purposes of illustration,the activity summary generation modules 220 a/220 b will be referred toas activity summary generation module 220 as they have similarfunctionality and function for a similar purpose. More detailsdescribing the features and functionalities of these modules will bediscussed further below in the detailed description of FIGS. 2 and 3.

The application server 107 may include similar components as the networkserver 101. Such components as they relate to the network server 101 aredescribed in the description of FIG. 2. The application server 107 mayprovide services to users of the system 100. The application server 107may be a third party application server that hosts other third partywebsites or third party applications. For example, the applicationserver 107 may host a website for user to listen to music, or may host awebsite for users to watch videos or play games. The application server107 is communicatively coupled to send information to the network server101 via the network 105. Such information sent to the network server 101may be information associated with user activity on the applicationserver 107.

The network 105 enables communications between user devices 115 a, 115b, the network server 101 and the application server 107. Thus, thenetwork 105 can include links using technologies such as Wi-Fi, Wi-Max,2G, Universal Mobile Telecommunications System (UMTS), 3G, Ethernet,802.11, integrated services digital network (ISDN), digital subscriberline (DSL), asynchronous transfer mode (ATM), InfiniBand, PCI ExpressAdvanced Switching, etc. Similarly, the networking protocols used on thenetwork 105 can include the transmission control protocol/Internetprotocol (TCP/IP), multi-protocol label switching (MPLS), the UserDatagram Protocol (UDP), the hypertext transport protocol (HTTP), thesimple mail transfer protocol (SMTP), the file transfer protocol (FTP),lightweight directory access protocol (LDAP), Code Division MultipleAccess (CDMA), Wideband Code Division Multiple Access (WCDMA), GlobalSystem for Mobile communications (GSM), High-Speed Downlink PacketAccess (HSDPA), etc. The data exchanged over the network 105 can berepresented using technologies and/or formats including the hypertextmarkup language (HTML), the extensible markup language (XML), etc. Inaddition, all or some of links can be encrypted using conventionalencryption technologies such as the secure sockets layer (SSL), SecureHTTP and/or virtual private networks (VPNs) or Internet Protocolsecurity (IPsec). In another implementation, the entities can use customand/or dedicated data communications technologies instead of, or inaddition to, the ones described above. Depending upon theimplementation, the network 105 can also include links to othernetworks.

In one implementation, the network 105 is a partially public or a whollypublic network such as the Internet. The network 105 can also be aprivate network or include one or more distinct or logical privatenetworks (e.g., virtual private networks, Wide Area Networks (“WAN”)and/or Local Area Networks (“LAN”)). Additionally, the communicationlinks to and from the network 105 can be wireline or wireless (i.e.,terrestrial- or satellite-based transceivers). In one implementation,the network 105 is an IP-based wide or metropolitan area network.

In the illustrated implementation, the user device 115 a is coupled tothe network 105 via signal line 108. The user 125 a can interact withthe user device. The user device 115 b is coupled to the network 105 viasignal line 112. The user device 115 n is couple to the network 105 asillustrated by signal line 114. The application server 107 iscommunicatively coupled to the network 105 via signal line 106. Thenetwork server 101 is communicatively coupled to the network 105 viasignal line 104. The network server 101 is also communicatively coupledto data storage 110 via signal line 102.

Data storage 110 may store data and information of users 125 a/125 b/125n of the system 100. Such stored information includes user profiles andother information identifying the users 125 a/125 b/125 n of the system100. Examples of information identifying users includes, but is notlimited to, the user's name, contact information, relationship status,likes, interests, links, education and employment history, location,etc. The information stored in data storage 110 may also include theuser's list of current and past friends and the user's activities withinthe system 100, such as anything the user posts within the system andany messages that the user sends to other users.

Data storage 110 may also store data and information of users 125 a/125b/125 n of the system 100 received from the application server 107. Forexample, after obtaining user consent, an application server 107 may bea music streaming application or website, which may send information todata storage 110 that identifies the songs or playlists that werelistened to by a user. Data storage 110 may also store multimedia suchas pictures or videos that were taken by users 125 a/125 b/125 n of thesystem 100.

A user device 115 a, 115 b, or 115 n may be an electronic computingdevice having a web browser for interacting with the network server 101via the network 105 and is used by user 125 a, 125 b, 125 n to accessinformation in the system 100. The user device 115 a, 115 b, 115 n canbe a computing device, for example, a laptop computer, a desktopcomputer, a tablet computer, a mobile telephone, a personal digitalassistant (PDA), a mobile email device, a portable game player, aportable music player, a television with one or more processors embeddedtherein or coupled thereto, or any other electronic device capable ofaccessing a network. A server can also be a computing device.

FIG. 2 is a block diagram illustrating an example of a network server101. As illustrated in FIG. 2, the network server 101 includes a networkadapter 202 coupled to a bus 204. Also coupled to the bus 204 are atleast one processor 206, memory 208, a graphics adapter 210, an inputdevice 212 and a storage device 214. In one implementation, thefunctionality of the bus 204 is provided by an interconnecting chipset.The network server 101 also includes a display 218, which is coupled tothe graphics adapter 210. As illustrated in FIG. 2, memory 208 includesa social network module 209 and an activity summary generation module220. The network server 101 may include or be communicatively coupled toan application service 109. Such application service 109 may provideother online services to users. The application service iscommunicatively coupled to the network server 101 and sends information,such as information related to user activity on the application server109, to the network server 101. More details describing thefunctionality and components of the activity summary generation module220 will be described below in the description for FIG. 3.

The processor 206 may be any general-purpose processor. The processor206 comprises an arithmetic logic unit, a microprocessor, a generalpurpose controller or some other processor array to performcomputations, provide electronic display signals to display 218. Theprocessor 206 is coupled to the bus 204 for communication with the othercomponents of the user device 115 a/115 b. Processor 206 processes datasignals and may comprise various computing architectures including acomplex instruction set computer (CISC) architecture, a reducedinstruction set computer (RISC) architecture, or an architectureimplementing a combination of instruction sets. Although only a singleprocessor is shown in FIG. 2, multiple processors may be included. Theuser device 115 a/115 b also includes an operating system executable bythe processor such as but not limited to WINDOWS®, MacOS X, Android, orUNIX® based operating systems.

The memory 208 holds instructions and data used by the processor 206.The instructions and/or data comprise code for performing any and/or allof the techniques described herein. The memory 208 may be a dynamicrandom access memory (DRAM) device, a static random access memory (SRAM)device, flash memory or some other memory device known in the art. Inone implementation, the memory 208 also includes a non-volatile memorysuch as a hard disk drive or flash drive for storing log information ona more permanent basis. The memory 208 is coupled by the bus 204 forcommunication with the other components of the user device 115 a/115 b.

The storage device 214 is any device capable of holding data, like ahard drive, compact disk read-only memory (CD-ROM), DVD, or asolid-state memory device. The storage device 214 is a non-volatilememory device or similar permanent storage device and media. The storagedevice 214 stores data and instructions for processor 206 and comprisesone or more devices including a hard disk drive, a floppy disk drive, aCD-ROM device, a DVD-ROM device, a DVD-RAM device, a DVD-RW device, aflash memory device, or some other mass storage device known in the art.

The input device 212 may be a mouse, track ball, or other type ofpointing device, and is used in combination with another input device212, such as a keyboard to input data into the network server 101. Thegraphics adapter 210 displays images and other information on thedisplay 218. The display 218 is a conventional type such as a liquidcrystal display (LCD) or any other similarly equipped display device,screen, or monitor. The display 218 represents any device equipped todisplay electronic images and data as described herein. The networkadapter 202 couples the network server 101 to a local or wide areanetwork.

In one implementation, the social network module 209 is software and/orroutines executable by a processor to control the interaction betweenthe network server 101, storage device 214 and the user devices 115 a,115 b, 115 n. An implementation of the social network module 209 allowsusers 125 a, 125 b, 125 n of user devices 115 a, 115 b, 115 n to performsocial functions between other users 125 a, 125 b, 125 n of user devices115 a, 115 b, 115 n within the system 100. In some implementations, thesocial network module 209 may provide the user with an option to opt-inor opt-out of requesting the user activity data from the applicationserver 107 or application service 109.

The activity summary generation module 220 is software and/or routinesthat, when executed by a processor, generate an activity summary to auser within the system 100. An activity summary may be a grouping ofuser activity information and content that represents the user'sactivity. In some implementations, the activity summary is based on theuser's activity. More details describing the functionality andcomponents of the activity summary generation module 220 will bedescribed below in the description for FIG. 3.

The network server 101 can have different and/or other components thanthose shown in FIG. 2. As is known in the art, the network server 101 isadapted to execute computer program modules for providing functionalitydescribed herein. As used herein, the term “module” encompasses it plainand ordinary meaning, including, but not limited to computer programlogic utilized to provide the specified functionality. Thus, a modulecan be implemented in hardware, firmware, and/or software. In oneimplementation, program modules are stored on the storage device 214,loaded into the memory 208, and executed by the processor 206.

Implementations of the entities described herein can include otherand/or different modules than the ones described here. In addition, thefunctionality attributed to the modules can be performed by other ordifferent modules in other implementations. Moreover, this descriptionoccasionally omits the term “module” for purposes of clarity andconvenience.

FIG. 3 is a block diagram illustrating examples of modules within theactivity summary generation module 220. As stated above, the activitysummary generation module 220 may be software and/or routines executableby the processor to generate activity summaries to a user of the system100. An activity summary may be a grouping of user activity informationand content that represents the user's activity. For example, theactivity summary may be a representation of songs that the user haslistened to that day. As another example, the activity summary may be arepresentation or summary of the user's activities while the user was onvacation. In some implementations, a particular activity may beassociated with a specific action and each activity may be differentactions. In such implementations, the activity summary may includemultiple actions of a user. The activity summary generation module 220may include a user activity information receiver module 302, acategorization module 304, a ranking module 306, a content selectionmodule 308, a segmentation module 310, an enrichment module 312, atitling module 314, and an output generation module 316. One or more ofthese modules may be implemented together and may be embodied as asingle module.

The activity summary generation module 220 may be configured tocommunicate with user devices 115 to receive input from the user devices115, application servers 107 and/or application services 109. Theactivity summary module 220 may be configured to communicate with theuser devices 115 to send activity summaries for display on the userdevices 115. The input received from the user devices 115, applicationservers 107 and/or application services 109 may include user activityinformation, which may be activity that the user has performed. The useractivity information including, but is not limited to, informationassociated with audio (for example, songs) that that user 125 haslistened to, pictures that the user has uploaded to the network server101, videos that the user has watched, text posts that the user hasinput into the network server 101, physical activities performed by theuser as determined from accelerometer data (e.g., walking, running,driving, flying), physical activities performed by the user asdetermined by social networking updates generated by the user (e.g.,“went clubbing today”), social networking activities of the user (e.g.,reading social networking content, liking social networking content),online activities of the user (e.g., researching a topic), non-onlineactivities of the user performed on a computing device, and otherinformation associated with the activities of the user 125. In someimplementations, the activity summary generation module 220 is alsoconfigured to communicate with storage device 214 and the data store 110to send user activity information for storage and retrieval. In someimplementations, the activity summary generation module 220 is alsoconfigured to communicate with the application server 107 to receiveuser activity on third party applications.

The user data receiver module 302 of the activity summary generationmodule 220 is software and/or routines for receiving a user activityinformation from a user device 115 and user activity information storedon storage device 214 or data storage 110. The user data receiver module302 may also receive user activity information from application servers107 and application services 109. The user activity information may beany information associated with the activities of the user when the useris interacting with the application server 107 or application services109. The user activity information may be information associated withaudio (for example, songs) that that user 125 has listened to, picturesthat the user has uploaded to the network server 101, videos that theuser has watched, text posts that the user has input into the networkserver 101, and other information associated with the activity of theuser 125. Additionally, user activity information may be informationreceived from the application server 107 or application service 109related to user activity on the application server 107 or applicationservice 109. For example, if the application server 107 hosts a websitethat allows users to listen to music or watch videos, the user activityinformation regarding songs listened to or videos watch on theapplication server 107 may be sent to and received by the user datareceiver module 302 of the activity summary generation module 220.

The categorization module 304 of the activity summary generation module220 is software and/or routines for categorizing the received useractivity information. The user activity information may be categorizedor indexed according to various categories or criteria. In some cases,the user activity information may be categorized according to a useridentifier, a verb identifier associated with the user activity, atarget associated with the user activity and a source. For example, auser identifier can be an anonymized identifier that uniquely identifiesa particular user 125 of the system 100. The verb identifier associatedwith the user activity may be predefined according to the particularuser activity. In some implementations, if a particular website or aparticular application service 109 or application server 107 providesthe user with the ability to perform more than one activity on thatwebsite, service or server, there may be more than one verb identifierassociated with that website, service or server, and each verbidentifier may be associated with a certain user activity. For example,if the user activity occurs on a website that hosts videos for viewing,a verb identifier associated with that website and activity may be“watch” or “watched” and the corresponding activity of watching videosthat occurs on that website would be associated with “watched.” Toextend the example, if the website also allows a user to listen tomusic, the verb associated with that corresponding activity of listeningto songs other audio files may be “listen” or “listened.” In thisexample, the website may be associated with various user activities andtherefore various corresponding verb identifiers. The target associatedwith the user activity identifies an item (such as a video in theprevious example) that was a target of the user's activity. The targetmay include a title and a Uniform Resource Locator (ULR) where thetarget can be found.

The user activity information from multiple servers 107 may have thesame associated verb. Similarly, user activity information from multipleapplication services 109 may have the same associated verb. For example,there may be multiple application servers 107 that allows users tolisten to music. In such a case, for user activities associated withthese application servers 107, the activity would be categorized usingthe same verb and therefore the activity would be categorized together,for example, under the verb “listened.” The categorization module 304may also group the categorized user activity. For example, thecategorization module 304 may aggregate the categorized based on thetype of activity.

The categorization module 304 may also group the categorized useractivity based on other criteria and these categorized groups mayinclude different types of user activity or content. The categorizationmodule 304 may also group the user activity information in accordancewith commonalities identified among the user activity information. Bygrouping the user activity information in accordance with commonalitiesidentified among the user activity information, the categorizationmodule 304 helps to organize the user activity or online activity intointeresting stories or comprehensive summaries of these activities. Thiscategorized and organized user activity represents experiences and maybe helpful in conveying the user's life experiences when the experiencesrelate to a common theme. The categorization module 304 may group theuser activity information based on relatedness (similar topic, similarcontent, etc.) of the user activity. Commonalities may include alocation common to the user activity. Commonalities may also include aninterest or topic common to the user activity. Groups of user activitycreated by the categorization module may therefore include differenttypes of user activity. For example, the categorization module 304 maycategorize and group together the user activity that the user performedwhile the user was in Japan. As such, in this example, the categorizedand grouped user activity, which eventually becomes output or isdisplayed as the activity summary, may include activity and content suchas the pictures the user posted of the user's trip to Japan, the placesthat the user visited while in Japan, and the items the user purchasedonline while the user was in Japan.

Further, the grouping and categorization performed by the categorizationmodule 304 may be based on session determination and whether the useractivity falls within a particular session. The categorization module304 may also determine sessions of user activity. The categorizationmodule 304 may determine sessions of user activity based on the time theuser activity was performed. The time may be based on a certain timeframe, for example, every six hours, or based on certain times of theday, for example, from 9:00 AM to 5:00 PM. The categorization module 304may determine sessions of user activity based on when the user signs offand/or signs in. The sign-on or sign-in may be associated with activityon the application server 107 or application service 109. The sessionmay also be based on other factors, e.g., relatedness of the useractivities. For example, relatively continuous user activities that arerelated to a particular topic or particular type of content can indicatea session.

The ranking module 306 of the activity summary generation module 220 maybe software and/or routines for ranking the categorized user activityinformation. The categorized user activity information may be ranked 406according to relevance to the user by the ranking module 306. Thecategorized user activity information may be ranked 406 according torelevance to the user's contacts by the ranking module 306.

The content selection module 308 of the activity summary generationmodule 220 may be software and/or routines for selecting the content tobe included in an activity summary of a user. Content selection mayinclude photo selection, as well as location determination and selectionof other content relevant to generating the activity summary. In someimplementations, the content selection module 308 of the activitysummary generation module 220 may receive user activity information fromthe user data receiver module 302 as well as multimedia data andsecondary data, such as location information or time information anddetermine which of that content to include in the activity summary. Insome implementations, the content selection may be based on a ranking ofthe content, the quality of the content or other aspects of the contentthat would render the content valuable for selection to be included inthe activity summary.

The segmentation module 310 of the activity summary generation module220 may be software and/or routines for analyzing the received useractivity and content, including received content, such as multimedia orother data or information and determining how the received contentrelates to each other. In some implementations, the segmentation module310 of the activity summary generation module 220 may receive useractivity information from the user data receiver module 302. In someimplementations, the segmentation module 310 of the activity summarygeneration module 220 may receive selected content from the contentselection module 308 of the activity summary generation module 220. Insome implementations, the segmentation module 310 may determinecommonalities between the content and segment the content based on thecommonalities. For example, in some implementations, the segmentationmodule 310 may determine that the receive content includes pictures thatwere taken in the same location and segment and group the content(pictures) based on that commonality.

The enrichment module 312 of the activity summary generation module 220may be software and/or routines for adding additional information to theselected content and multimedia. In some implementations, contentrecognition is performed on the multimedia. Based on the recognizedcontent and additional search of a database may be performed to obtainadditional information about the recognized content. The additionalinformation may be added to the content to further enhance itsinformational quality. For example, a recognition of a media item may beperformed and a picture of the Golden Gate Bridge may be found. Theenrichment module 312 may search a database of information and findadditional information about the Golden Gate Bridge, for example, howlong it is. The enrichment module 312 may then add that information inthe form of text to the selected media as part of the activity summary.

In some implementations, the enrichment module 312 communicates with thecontent selection module 308 to add the information to the selectedcontent. In some implementations, the enrichment module 312 alsodetermines other information from metadata that may be associated withthe content and add additional information based on the metadata. Insome implementations, the enrichment module 312 of the activity summarygeneration module 220 may look at location information and determine aspecific named location associated with the determined location. Forexample, the enrichment module 312 may determine that the contentreceived, for example, pictures taken were taken at a specific shoppingcenter and specifically at a certain store. The enrichment module 312may then use that information and add a map location of the store to theactivity summary to enhance the content.

The titling module 314 of the activity summary generation module 220 maybe software and/or routines for determining and assigning a title to theselected content of the activity summary. In some implementations, thetitling module 314 of the activity summary generation module 220 mayanalyze the content of the activity summary to determine words and/orphrases to best summarize the content of the activity summary. In someimplementations, the titling module 314 of the activity summarygeneration module 220 determines a time frame for when the activity hastaken place and determines a general description to describe that timeframe. For example, if the activity took place from Friday at 6:00 pmand ended on Sunday at 12:00 pm, the titling module 314 of the activitysummary generation module 220 may assign a title “The Weekend” to theactivity summary. In another implementation, the titling module 314 ofthe activity summary generation module 220 determines the generallocation of the activity summary. For example, the titling module 314 ofthe activity summary generation module 220 may assign a title “SanFrancisco” to the activity summary if it determines that the activityhappened mostly in San Francisco. In other implementations, the titlingmodule 314 of the activity summary generation module 220 may use boththe determined location and the time frame to determine and assign atitle. For example, if the titling module 314 of the activity summarygeneration module 220 determines that the activity occurred from Fridayat 6:00 pm and ended on Sunday at 12:00 pm and took place generally inSan Francisco, the titling module 314 of the activity summary generationmodule 220 may assign the title “Weekend in San Francisco” to theactivity summary.

In yet other implementations, the titling module 314 of the activitysummary generation module 220 may recognize that the time and/or contentdescribed in the story closely matches an event described on the user'scalendar (e.g. “Tom's birthday party”) or an event the user was invitedto and may determine and assign a title also based on the calendar itemor event.

In another implementation, the titling module 314 may generate titles ina language-agnostic way so that the title can be translated intowhatever language the recipient of the story speaks. As an example, insuch implementations, a template may be generated that may be: “Time:[weekend], locations: [San Francisco, Napa],” and based on the generatedtemplate, the initial title may be translated and re-assigned to a morenaturally-sounding title in whatever language the user normally speaks.Furthermore, in some implementations, the title may be based on theaggregated activity information, for example, based on videos the userwatched or music the user listened to. In such examples, in someimplementations, the titling module 314 may determine and assign a title“New music you listened to last month” or “Top artists you listened tolast week” based on aggregated activity information.

The output generation module 316 of the activity summary generationmodule 220 is software and/or routines for generating the activitysummaries based on the categorized user activity information. The outputgeneration module 316 receives information from the categorizationmodule 304 and the ranking module 306 and generates activity summariesbased on the categorized user activity information. In someimplementations, the user may share the generated activity summary withother users, for example, by posting the activity summary on the user'ssocial network portal. The post may be a public post where other userscan view the activity summary. The post may also be a private post whereselected users may view the activity summary.

Referring now to FIG. 4, an example method 400 for generating activitysummaries for a user 125 of the system 100 will be described. In someimplementations, the method 400 is performed by the components of thenetwork server 101, and specifically, by the components of the activitysummary generation module 220. In one implementation, the method 400begins when user activity information is received 402 by the useractivity information receiver module 302. The user activity informationis then categorized 404 by the categorization module 304. In someimplementations, the categorized user activity information is ranked 406according to relevance to the user by the ranking module 306. In someimplementations, the categorized user activity information is ranked 406according to relevance to the user's contacts by the ranking module 306.Activity summaries are then generated 408 by the output generationmodule 316 of the activity summary generation module 220. The activitysummary includes the categorized user activity information. The activitysummary is sent 410 for display on a user device 115 of a user 125. Insome implementations, the user may share the generated activity summarywith other users, for example, by posting the activity summary on theuser's social network portal. The post may be a public post where otherusers can view the activity summary. The post may also be a private postwhere selected users may view the activity summary.

Referring now to FIG. 5, another example method 500 for generatingactivity summaries to a user 125 of the system 100 will be described. Insome implementations, the method 500 is performed by the components ofthe network server 101, and specifically, by the components of theactivity summary generation module 220. In one implementation, themethod 500 begins when user activity information is received 502 by theuser activity information receiver module 302. The user activityinformation may be information associated with audio (for example,songs) that that user 125 has listened to, pictures that the user hasuploaded to the network server 101, videos that the user has watched,text posts that the user has input into the network server 101, andother information associated with the activity of the user 125.Additionally, user activity information may be information received fromthe application server 107 or application service 109 related to useractivity on the application server 107 or application service 109. Insome implementations, the user activity information may also be photosthat the user has taken and location information associated with thosephotos. In some implementations, the activity information may alsoinclude the user's location information (or location information of theuser's device), the user's calendar information, or information relatedto events that the user will be attending or has attended, or otherinformation related to the user's activity. For example, if theapplication server 107 hosts a website that allows users to listen tomusic or watch videos, the user activity information regarding songslistened to or videos watch on the application server 107 may be sent toand received by the user data receiver module 302 of the activitysummary generation module 220. In one implementation, a user's searchhistory may also serve as a source for activity information.

The user activity information is then categorized 504 by thecategorization module 304. In some implementations, the user activityinformation may be categorized according to a user identifier, a verbidentifier associated with the user activity, a target associated withthe user activity and a source. For example, a user identifier can be ananonymized identifier that uniquely identifies a particular user 125 ofthe system 100. The verb identifier associated with the user activitymay be predefined according to the particular user activity. In someimplementations, if a particular website or a particular applicationservice 109 or application server 107 provides the user with the abilityto perform more than one activity on that website, service or server,there may be more than one verb identifier associated with that website,service or server, and each verb identifier may be associated with acertain user activity. For example, if the user activity occurs on awebsite that hosts videos for viewing, a verb identifier associated withthat website and activity may be “watch” or “watched” and thecorresponding activity of watching videos that occurs on that websitewould be associated with “watched.” In some implementations, the useractivity information may also be categorized according to

The categorization module 304 also groups 506 the categorized useractivity information in accordance with commonalities identified amongthe user activity information. By grouping the user activity informationin accordance with commonalities identified among the user activityinformation, the categorization module 304 helps to organize the useractivity or online activity into interesting stories or comprehensivesummaries of these activities. This categorized and organized useractivity represents online experiences and may be helpful in conveyingthe user's life experiences when the experiences relate to a commontheme. The categorization module 304 may group the user activityinformation. The user activity information may be grouped 506 based onrelatedness (similar topic, similar content, etc.) of the user activity.Commonalities may include a location common to the user activity.Commonalities may also include an interest or topic common to the useractivity. Groups of user activity created by the categorization modulemay therefore include different types of user activity. For example, thecategorization module 304 may categorize and group together the useractivity according to activity that the user performed online while theuser was in Japan. In this example, categorized and grouped useractivity, which eventually becomes output or displayed as the activitysummary, may include activity and content such as the pictures the userposted of the user's trip to Japan, the places that the user visitedwhile in Japan, and the items the user purchased online while the userwas in Japan.

In some implementations, the categorization module 304 also groups 506the categorized user activity information in accordance withcommonalities identified among the user activity information todetermine the beginning of a specific group in order to create a storyfrom that grouping. In one implementation, a change in a user's locationsignals the beginning of a story. In one implementation, new photossignal the beginning of a story. In another implementation, a change inthe user's location from their usual location signals the beginning of astory. In some implementations the user's usual location may be aspecifically identified location (for example, a defined home location,or an identified work location). In other implementations, the user'susual location may be inferred from the user's location history, or frompreviously geo-tagged photos (e.g. where the user tends to spend most ofhis or her time).

In some implementations, the categorized user activity information isranked 508 according to relevance to the user by the ranking module 306.In some implementations, the categorized user activity information isranked 508 according to relevance to the user's contacts by the rankingmodule 306.

The output generation module 316 determines 510 when the groupings arecomplete. Certain signals or activity information are monitored andanalyzed to determine the completion of a grouping or ending of a story.Specifically, certain signals such as photos, location, calendar, eventsand other user's information (which is linked to the first user) serveas sources for activity information and indications of the completion ofa grouping or ending of a story. In one implementation, a user's searchhistory may also serve as a source for activity information. In oneimplementation, a change in a user's location signals the ending of astory. In one implementation, a time limit may signal the ending of astory.

Activity summaries are then generated 512 by the output generationmodule 316 of the activity summary generation module 220. In someimplementations, a notification that activity information is beingreceived, processed and categorized is generated. In suchimplementations, such notifications may include a notification that astory is being created. In other implementations, a notification of thecompleted version activity summary is generated. The activity summaryincludes the categorized user activity information. The activity summaryor notification is sent 514 for display on a user device 115 of a user125. In some implementations, the user may share the generated activitysummary with other users, for example, by posting the activity summaryon the user's social network portal. The post may be a public post whereother users can view the activity summary. The post may also be aprivate post where selected users may view the activity summary.

FIG. 6 is a flow chart 600 illustrating another example method forgenerating activity summaries. In one implementation, the method 600begins when user activity information is received 602 by the useractivity information receiver module 302. The user activity informationmay be information associated with audio (for example, songs) that thatuser 125 has listened to, pictures that the user has uploaded to thenetwork server 101, videos that the user has watched, text posts thatthe user has input into the network server 101, and other informationassociated with the activity of the user 125. Additionally, useractivity information may be information received from the applicationserver 107 or application service 109 related to user activity on theapplication server 107 or application service 109. In someimplementations, the user activity information may also be photos thatthe user has taken and location information associated with thosephotos. In some implementations, the activity information may alsoinclude the user's location information (or location information of theuser's device), the user's calendar information, or information relatedto events that the user will be attending or has attended, or otherinformation related to the user's online activity.

The user data receiver module 302 waits 604 a predefined period of time.This allows enough content and data to be received to be able toadequately create an activity summary. In some implementations, the userdata receiver module 302 waits several hours, for example, one hour, twohours, three hours, four hours, etc. A determination 606 is performed todetermine whether new information has been received. For example,whether the user has taken any new pictures within the predefined periodof time, If new information has been received (606—Yes), the datareceiver module 302 continues to receive 602 user activity information.If new information has not been received (606—No), analysis of thereceived information is performed 610. A more detailed descriptiondescribing step 610 will be described below with regard to FIG. 7.Finally, an activity summary (or story) is generated 612 and sent fordisplay.

FIG. 7 is a flow chart 700 illustrating an example method for performingdata analysis to create an activity summary. Data is processed 702 bythe user activity information receiver module 302. Once enough data isreceived and processed, the data is analyzed and segmented 704 todetermine and create an activity summary or story. In someimplementations, segmentation is performed based on a commonality.Content is selected 706 to determine what content is included in theactivity summary. The selection of the content includes selection ofmultimedia items, such as pictures and videos. Secondary information,such as user activity information or location information is analyzed708. Other secondary information may include metadata associated withthe content or multimedia item, or other information associated with thecontent or multimedia item. Once the content is selected and analyzed,the content is enriched 710 or enhanced with additional informationbased on the analysis of the secondary information. Using the selectedcontent, a visual representation of generated and a title is assigned712 that identifies the representation.

FIG. 8 is an example graphical user interface 800 showing a title pageof an activity summary. The graphical user interface 800 includes atitle area 802. The graphical user interface 800 also includes a usericon 804. The graphical user interface 800 may also include a titlepicture 808.

FIG. 9 is an example graphical user interface 900 showing another pageof an activity summary. The graphical user interface 900 includes alocation identifier 902 and a comment 904. The comment 904 may beentered by the user and/or viewers of the activity summary. According tothis implementation, the graphical user interface also includes a routemap 906. The route map 906 illustrates the location or route associatedwith the particular activity summary.

FIG. 10 is an example graphical user interface 1000 showing yet anotherpage of an activity summary. The graphical user interface 1000 includesa location identifier 1002. For example, this location identifier may beadded by the enrichment module 312. The graphical user interface 1000also includes selected multimedia 1004, 1006, 1008, and 1010. Thegraphical user interface 1000 also includes a time interstitial 1012.The time interstitial 1012 indicates the relative passing of time inrelation to the previous selected content that was displayed. Asillustrated in this example, the time interstitial 1012 indicates anamount of time that has passed from the previous displayed content.According to another implementation, the time interstitial 1012 mayindicated the exact time of the displayed selected media. In otherimplementations, the time interstitial 1012 may indicate day or date.

FIG. 11 shows the continuation of graphical user interface 1000 as thedisplay moves along the timeline of the activity summary. In FIG. 11,the graphical user interface 1000 includes selected multimedia 1102,1104, 1106, 1108, and 1110. In FIG. 11, the graphical user interface1000 also includes a comment field 1112 where a user may enter text. Theactivity summary may be laid out in a horizontally or verticallyscrollable configuration.

FIG. 12 is an example graphical user interface 1200 showing anotherexample of a title page of an activity summary. Similar to the graphicaluser interface 800 depicted in FIG. 8, the graphical user interface 1200includes a title area 1202. The graphical user interface 1200 alsoincludes user icons 1204 and 1206. The graphical user interface 800 mayalso include a title picture 1208.

FIG. 13 is an example graphical user interface 1300 showing another pageof an activity summary. The graphical user interface 1300 includes alocation identifier 1302. The graphical user interface 1300 alsoincludes selected multimedia 1304 and 1306. The graphical user interface1300 also includes a time interstitial 1310. The graphical userinterface 1300 also includes a comment field 1308 where a user may entertext.

FIG. 14 shows another example of graphical user interface 1400. Thegraphical user interface 1400 includes a location identifier 1402. Thegraphical user interface 1400 also includes selected multimedia 1404 and1406. The graphical user interface 1400 also includes a timeinterstitial 11410. In this illustration, the graphical user interface1400 shows the comment field 1410 that was automatically generated withadditional information by the enrichment module 312. In this example,there is a well-known landmark in the selected multimedia item 1406,which was identified and additional information added to enhance andenrich the display of information in the activity summary.

FIG. 15 shows the continuation of graphical user interface 1400 as thedisplay moves along the timeline of the activity summary. In FIG. 15,the graphical user interface 1400 includes selected multimedia items.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether programs orfeatures collect user information (e.g., information about a user'ssocial network, social actions or activities, profession, a user'spreferences, or a user's current location), or to control whether and/orhow to receive content from the content server that may be more relevantto the user. In addition, certain data may be treated in one or moreways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by a content server.

The foregoing description of the implementations of the presentdisclosure has been presented for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit the presentimplementations to the precise form disclosed. Many modifications andvariations are possible in light of the above teaching. It is intendedthat the scope of the present implementations be limited not by thisdetailed description, but rather by the claims of this application. Aswill be understood by those familiar with the art, the presentdisclosure may be embodied in other specific forms without departingfrom the spirit or essential characteristics thereof. Likewise, theparticular naming and division of the modules, routines, features,attributes, methodologies and other aspects are not mandatory orsignificant, and the mechanisms that implement the presentimplementations or their features may have different names, divisionsand/or formats. Furthermore, as will be apparent to one of ordinaryskill in the relevant art, the modules, routines, features, attributes,methodologies and other aspects of the present implementations can beimplemented as software, hardware, firmware or any combination of thethree. Also, wherever a component, an example of which is a module, ofthe present implementations are implemented as software, the componentcan be implemented as a standalone program, as part of a larger program,as a plurality of separate programs, as a statically or dynamicallylinked library, as a kernel loadable module, as a device driver, and/orin every and any other way known now or in the future to those ofordinary skill in the art of computer programming. Additionally, thepresent implementations are in no way limited to implementation in anyspecific programming language, or for any specific operating system orenvironment. Accordingly, the disclosure is intended to be illustrative,but not limiting, of the scope of the present implementations, which isset forth in the following claims.

What is claimed is:
 1. A computer-implemented method comprising:receiving, using one or more computing devices, activity information,wherein the activity information includes user activity information;segmenting the activity information to create an activity summary;generating a visual representation of the activity summary; and sending,using the one or more computing devices, the visual representation ofthe activity summary for display.
 2. The computer-implemented method ofclaim 1 wherein the activity information also includes at least onecontent item.
 3. The computer-implemented method of claim 1 whereinsegmenting the activity information is based on a categorization of theactivity information based on a commonality.
 4. The computer-implementedmethod of claim 1 wherein the at least one content item is a multimediaitem.
 5. The computer-implemented method of claim 4 wherein generatingthe visual representation of the activity summary utilizes the at leastone content item.
 6. A computer-implemented method comprising:receiving, using one or more computing devices, activity information,wherein the activity information includes user activity information andat least one content item; categorizing the activity information;grouping the categorized activity information based on a commonality;and generating an activity summary, the activity summary including thegrouped user activity information; generating a visual representation ofthe activity summary; and sending, using the one or more computingdevices, the visual representation of the activity summary for display.7. The computer-implemented method of claim 6, further comprising:ranking the grouped activity information based on relevance to the user.8. The computer-implemented method of claim 6, further comprising:determining when a group is complete.
 9. A computer program productcomprising a non-transitory computer usable medium including a computerreadable program, wherein the computer readable program when executed ona computer causes the computer to: receive activity information, whereinthe activity information includes user activity information; segment theactivity information to create an activity summary; generate a visualrepresentation of the activity summary; and send the visualrepresentation of the activity summary for display.
 10. The computerprogram product of claim 9 wherein the activity information alsoincludes at least one content item.
 11. The computer program product ofclaim 9 wherein segmenting the activity information is based on acategorization of the activity information based on a commonality. 12.The computer program product of claim 9 wherein the at least one contentitem is a multimedia item.
 13. The computer program product of claim 12wherein generating the visual representation of the activity summaryutilizes the at least one content item.
 14. A system comprising: aprocessor; and a memory storing instructions that, when executed, causethe system to: receive activity information, wherein the activityinformation includes user activity information; segment the activityinformation to create an activity summary; generate a visualrepresentation of the activity summary; and send the visualrepresentation of the activity summary for display.
 15. The system ofclaim 14 wherein the activity information also includes at least onecontent item.
 16. The system of claim 14 wherein segmenting the activityinformation is based on a categorization of the activity informationbased on a commonality.
 17. The system of claim 14 wherein the at leastone content item is a multimedia item.
 18. The system of claim 17wherein generating the visual representation of the activity summaryutilizes the at least one content item.